Gender Neutral or Responsive?

2019 Eastern Economics Association RN Poulson

Risk Assessments Drive Decisions

Risk & Juvenile Justice

  • Structured risk assessment tools estimate risk to reoffend.

  • Based on statistical analysis of theoretically based factors about a youth’s social situation and delinquency history.

  • Research shows outcomes are maximized.

  • Adopted by all states in U.S. and numerous countries throughout the world.

Utah as an Example

  • Risk assessments guide important decisions
    • Diversion from Juvenile Court.
    • Intensity & length of programming.
    • Placement, including in and out of home.
    • Criteria for ending contact with system.
  • Wide range of costs of services as low as $11 (Probation) to $1,224 (Adult Living for Transitional Achievement) per youth per day.

  • Policies shaped by risk assessment data.

Gender Responsivity in Juvenile Justice

Are Girls Different?

  • High level of self-reported trauma, abuse, and mental health.

  • Society and system tend to have a different view (protect & punish).

  • Challenges from male dominated society.

  • On average, come to the system via different pathways than boys.

Neutrality Critiques

  • Research is male focused.

  • Theory differences in programming effectiveness.
  • Girls punished more severely than boys for certain behaviors.

  • Different risk factors and criminogenic needs.

    • School, neighborhood effects (i.e., poverty) & religious affiliations
    • Parental authority & friends
    • Abuse, neglect, running away/kicked out of home, mental health, drugs/alcohol use.
  • Belief that girls are less likely to reoffend.

Gendered Critiques

  • Drivers of delinquency do not differ.

  • All youths should receive a responsive approach that accounts for trauma, mental health, abuse, neglect, and substance use problems.

  • Focus is on individual, not on institutional or structural change.
  • Gendered programming may reinforce socially constructed differences.

  • May increase over representation of males.

Gender-Specific Assessments

  • Adjust cut points that define risk categories so that fewer female offenders score as high risk.

  • Separate items and scoring mechanisms.

  • Incorporate gender as a predictor/scoring item.

Utah’s Approach

  • Pre-Screen Risk Assessment (PSRA) \(\Rightarrow\) gender specific.

  • All youths referred to the Juvenile Court.

  • Level of involvement and intensity of service.

Methods

Bayesian Additive Regression Trees

  • Chipman, George, and McCulloch (2010)

  • Bayesian perspective and machine learning techniques

  • Sum-of-trees model equipped with a “regularization prior”

  • Suited for non-experimental context where decision boundaries and correct predictors and functional form are unknown.

Sum-of-Trees

Basic Decision Tree

\(g(x;T,M)\)

Posterior Estimation

\(p(T,M,Z|X,Y)\propto p(Y|Z)p(Z|X,T,M)[\Pi_j\Pi_i p(\mu_{ij}|T_j)p(T_j)]\)

Ultimate goal generate a large number of trees given the observed data to estimate \(p(T,M,Z|X,Y)\)
\(\Downarrow\)

Likelihood Function
\(p(Y|Z)p(Z|X,T,M)\)

Regularization Prior
\(p(\mu_{ij}|T_j)\) prior distribution of the terminal node parameters to \(\Uparrow\) probability \(E(Y|x)\) in \(y_{min}\) and \(y_{max}\).

\(p(T_j)\) prior on \(T_j\) tree & includes three considerations: tree size, selection of predictors, and selection of values for splitting rules.

Posterior Sampling
Large parameter space \(\Rightarrow\) intractable calculations \(\Rightarrow\) MCMC algorithm comprised of Gibbs sampler to sample from posterior \(\Rightarrow\) gravitate toward regions of high posterior probability.
\(\Downarrow\)

Bayesian inferences about the estimation of f(x), predictions of y, posterior uncertainty, and the marginal effect of one or more predictors on the response. Model free variable selection.

Sample & Variables

  • Youths who received PSRA 1st time July 1, 2008 to June 30, 2014 (n = 15,244).

  • Girls \(\approx\) 31% (n = 4,682).

  • Delinquency and social history predictors.

  • Felony or misdemeanor level adjudication within one year \(\Rightarrow\) recidivism outcome.

Delinquency History

Boys tend to have more convictions for felony and weapon offenses and higher rate of detention admissions. Overall, the summary statistics suggest very little difference.
Girls
Boys
mean sd median min max mean sd median min max
Age 2.33 1.31 3 0 4 2.59 1.21 3 0 4
Felonies 0.26 0.75 0 0 6 0.54 1.08 0 0 6
Mis. 0.42 0.73 0 0 3 0.49 0.78 0 0 3
PersonFel. 0.06 0.36 0 0 4 0.22 0.72 0 0 4
Weapons 0.02 0.13 0 0 1 0.06 0.23 0 0 1
PersonMis. 0.19 0.46 0 0 2 0.20 0.47 0 0 2
Detention 0.17 0.44 0 0 3 0.22 0.48 0 0 3
JJS Custody 0.01 0.16 0 0 4 0.01 0.15 0 0 4
Escapes 0.00 0.08 0 0 2 0.00 0.05 0 0 1
FTA 0.02 0.16 0 0 2 0.01 0.13 0 0 2

Social History

Administrators of the PSRA selected scores that indicate girls in the sample have higher rates of social history problems in many areas their male peers. However, similar rates were found for school, child welfare, and mental health.
Girls
Boys
mean sd median min max mean sd median min max
School 1.13 0.84 1 0 2 1.12 0.83 1 0 2
Friends 0.95 0.68 1 0 3 0.89 0.71 1 0 3
Child Welfare 0.09 0.28 0 0 1 0.08 0.26 0 0 1
Runaway 0.30 0.66 0 0 2 0.16 0.49 0 0 2
Household 0.21 0.41 0 0 1 0.17 0.38 0 0 1
Compliance 0.59 0.72 0 0 2 0.49 0.68 0 0 2
Substances 0.60 0.91 0 0 2 0.56 0.90 0 0 2
Abuse/Neglect 0.32 0.60 0 0 2 0.23 0.57 0 0 2
Mental Health 0.16 0.37 0 0 1 0.14 0.35 0 0 1

Procedure

  1. Divide data into training and test groups.

  2. Build a gender neutral and gender specific model using BART.

  3. Estimate predicted probability of reoffending for each observation in the test data using each BART model.

  4. Compare predictive validity indicators.

  5. Explore variable importance for girls vs. boys.

Results

Risk Level - Girls

Risk Level - Boys

Delinquency History

Similar Results for Girls and Boys
Girls Boys
Age
Felonies X X
Misdemeanors
Person fel. X
Weapons X X
Person misdemeanors X X
Detention
JJS Custody X X
Escapes X X
FTA Warrants X X

Social History

Similar Results for Girls and Boys
Girls Boys
School X
Friends
Child Welfare X X
Runaway
Household X X
Parent Compliance
Substance Use X X
Abuse/Neglect X X
Mental Health X X

Predictive Validity

Similar AUCs Across Risk Assessment Types. Gender Neutral type slightly outperforms according to Somers’ d.
Gender Neutral Gender Specific
Girls
AUC 0.6797 0.6767
Somers’ d 0.3122 0.2909
Boys
AUC 0.6948 0.6954
Somers’ d 0.3448 0.3311

Discussion

Implications

  • Utah may want to move toward gender neutral assessment.

  • Some girls may be higher risk and some boys may be lower risk than currently predicted.

  • No evidence that there is substantial difference between predictors across gender.

Limitations

  • Youth outcomes influenced by system response to PSRA risk level.
  • Risk factors focus on internal factors, not structural/societal.

  • Non-experimental context.

  • Does not address programming questions.

  • Only included measures of discrimination, not calibration.

References


Andrews, D. A. (2001). Principles of Effective Correctional Programs. In L. Motiuk, & R. C. Serin, Compendium 2000 on Effective Correctional Programming. Ottawa: Correctional Services of Canada.

Andrews, D. A., & Dowden, C. (2005). Managing Correctional Treatment for Reduced Recidivism: A Meta-Analytic Review of Programme Integrity. Legal and Criminal Psychology,, 173-187.

Andrews, D. A., & Dowden, C. (2006). Risk Principle of Case Classificatoin in Correctional Treatment: A Meta-Analytic Investigation. International Journal of Offender Therapy and Comparative Criminolog, 88-100.

Andrews, D. A., & Dowden, C. (2007). The Risk-Need- Responsivity Model of Assessment and Human Service in Prevention and Corrections: Crime- Prevention Jurisprudence. Canadian Journal of Criminology and Criminal Justice, 439-464.

Andrews, D. A., Bonta, J., & Hoge, R. D. (1990). Classification of Effective Rehabilitation: Rediscovering Psychology. Criminal Justice and Behavior, 19-52.

Andrews, D. A., Bonta, J., & Wormith, J. S. (2006). The Recent Past and Near Future of Risk Assessment. Crime & Delinquency, 7-27.

Aos, S., Barnoski, R., & Lieb, R. (2006). Evidence-Based Adult Corrections Programs: What Works and What Does Not. Olympia: Washington State Institute for Public Policy.

Baio, G. (2013). Bayesian Methods in Health Economics. Boca Rotan: Taylor & Francis Group.

References


Baird, C. (2009). A Question of Evidence: A Critique of Risk Assessment Models Used in the Justice System. Madison: National Council of Crime and Delinquency.

Baird, C., Heinz, R., & Bemus, B. (1979). The Wisonsin Case Classification Staff Deployment Project: Two-Year Follow-Up Report. Madison, WI: Wisconsin Division of Correnctions.

Barnoski, R. (2004). Assessing Risk for Re-Offense: Validating the Washington State Juvenile Court Assessment. Olympia: Washington State Institute for Public Policy.

Berk, R. (2012). Criminal Justice Forecasts of Risk: A Machine Learning Approach. New York: Springer.

Bitna, K., Merlo, A. V., & Benekos, P. J. (2013). Effective Correctional Intervention Programmes for Juveniles: Review and Synthesis of Meta-Analytic Evidence. 169-189.

Blattenberger, G., & Fowles, R. (1995). Road Closures: Combining Data and Expert Opinion. In C. Gatsonis, J. S. Hodges, R. E. Kass, & N. D. Singpurwalla, Case Studies in Bayesian Statistics, Volume II. New York: Springer.

Bleich, J., & Berk, R. (2015). Extensions and Applications of Ensemble-of-Trees Methods in Machine Learning. Doctoral Dissertation. ProQuest.

Bleich, J., Kapelener, A., George, E. I., & Jensen, S. T. (2014). Variable Selection for BART: An Application to Gene Regulation. The Annals of Applied Statistics, 1750-1781.

References


Bonta, J., & Andrews, D. A. (2017). The Psychology of Criminal Conduct (6th ed.). New York: Taylor & Francis.

Bonta, J., & Wormith, J. S. (2013). Applying the Risk-Need-Responsivity Principles to Offender Assessment. In L. A. Craig, T. A. Grannon, & D. Louise, What Works in Offender Rehabilitation (pp. 71-93). Malden: John Wiley & Sons.

Bonta, J., Rugge, T., Scott, T., Bourgen, G., & Yessine, A. (2008). Exploring the Black Box of Community Supervision. Journal of Offender Rehabilitation, 248-270.

Chipman, H. A., George, E. I., & McCulloch, R. E. (2010). BART: Bayesian Additive Regression Trees. The Annals of Applied Statistics, 266-298.

Chipman, H., & Rob, M. (n.d.). BayesTree: Bayesian additive regression trees. Retrieved from https://CRAN.R-project.org/package=BayesTree

Chipman, H., George, E., Lemp, J., & McCulloch, R. (2010). Bayesian Flexible Modeling of Trip Duration. Transportation Research Part B, 686-698.

DeWitt, J. R., Poulson, R. N., & Wetherly, E. (2016). The Utah Pre-Screen Risk Assessment (PSRA) and the Protective and Risk Assessment (PRA) Validation Study II. Salt Lake City: Utah Adminstrative Office of the Courts and Division of Juvenile Justice Services.

DeWitt, J., & Lizon, R. (2008). The Utah Pre-Screen Risk Assessment (PSRA) and the Protective and Risk Assessment (PRA) Validation Study. Salt Lake City, Utah: Utah Adminstrative Office of the Courts and Division of Juvenile Justice Services.

References


Dowden, C., & Andrews, D. A. (2004). The Importance of Staff Practice in Delivering Effective Correctional Treatment: A Meta-Analytic Review of Core Correctional Practice. nternational Journal of Offender Therapy and Comparative Criminology, 203-214.

Freedman, D. A. (2010). Statistical Models and Causal Inference: A Dialogue with the Social Sciences. (D. Collier, J. S. Sekhon, & P. B. Stark, Eds.) New York: Cambridge University Press.

Friedman, J. H. (2001). Greedy Function Approximation: A Gradient Boosting Machine. Annals of Statistics, 1189-1232.

Gatti, U., Tremblay, R. E., & Vitaro, F. (2009). Iatrogenic Effect of Juvenile Justice. Child Psychology and Psychiatry, 991-998.

Gendreau, P. (1996). Principles of Effective Intervention With Offenders. In A. Harland, Choosing Correctional Options That Work: Defining the Demand and Evaluating the Suppl (pp. 117-130). Thousand Oaks: Sage Publications.

Gendreau, P., French, S. A., & Taylor, A. (Revised 2002). What Works (What Doesn’t Work). Invited Submission to the International Community Corrections Monograph Series Project.

Goodkind, Sara. (2005). Gender-Specific Services in the Juvenile Justice System: A Critical Examination. Affilia-journal of Women and Social Work - AFFILIA J WOMEN SOC WORK. 20. 52-70. 10.1177/0886109904272061.

References


Grove, W. M., Zald, D. H., Lebow, S. B., Snitz, E. B., & Nelson, C. (2000). Clinical Versus Mechanical Prediction: A Meta-Analysis. Psychological Assessment, 19-30.

Hamilton, Z., Kigerl, A., Campagna, M., Barnoski, R., Lee, S., Van Wormer, J., & Block, L. (2016). Designed to Fit: The Development and Validation of the STRONG-R Recidivism Risk Assessment. Criminal Justice and Behavior, 230-263.

Harris, A., Evans, H., & Beckett, K. (2010). Drawing Blood for Stones: Legal Debt and Social Inequality in the Contemporary United States. American Journal of Sociology, 1753-1799.

Harris, A., Evans, H., & Beckett, K. (2010). Drawing Blood from Stones: Legal Debt and Social Inequality in the Contemporary United States. American Journal of Sociology, 1753-1799.

Harris, G. T., M., E. R., Quinsey, V. L., Lalumiere, M. L., Boer, D., & Lang, C. (2003). A Mulitsite Comparison of Actuarial Risk Instruments for Sex Offenders. Psychological Assessment.

Harris, G. T., Rice, M. E., Quinsey, V. L., Lalumiere, M. L., Boer, D., & Lang, C. (2003). A Mulitsite Comparison of Actuarial Risk Instruments for Sex Offenders. Psychological Assessment.

Hastie, T., Tibshirani, R., & Friedman, J. (2017). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer .

Hill, B. M. (1986). Some Subjective Bayesian Considerations in the Selection of Models. Econometric Reviews, 191-246.

References


Homan, B., & Ziedenberg, J. (2007). The Dangers of Detention: The Impact of Incarcerating Youth in Detention and Other Secure Facilities. Retrieved March 2016, from http://www.aecf.org/resources/2014-juvenile-detention-alternatives-initiative-progress-report/: http://www.justicepolicy.org/images/upload/06-11_rep_dangersofdetention_jj.pdf

Howell, J. C. (2003). Preventing & Reducing Juvenile Delinquency. Thousand Oaks: Sage Pulbications.

Juvenile Justice Oversight Committee. (n.d.). HB 239 - Juvenile Justice Amendments. Retrieved March 2018, from Utah Board of Juvenile Justice, Utah Commission on Criminal and Juvenile Justice: https://justice.utah.gov/Juvenile/hb239.html

Kapelener, A., & Bleich, J. (2016). bartMachine: Machine Learning with Bayesian Additive Regression Trees. Journal of Statistical Software, 1-40.

Kennealy, P. J., Skeem, J. L., Manchak, S. M., & Louden, J. E. (2012). Firm, Fair, and Caring Officer- Offender Relationships Protect Against Supervision Failure. Law and Human Behavior, 496-505.

KiDeuk, K., & Duwe, G. (2016). Improving the Performance of Risk Assessments. In F. Taxman, Handbook on Risk and Need Assessment: Theory and Practice (pp. 114-139). Routledge.

Ladenberger, N. A., & Lipsey, M. W. (2005). A Meta- Analysis of Factors Associated with Effective Treatment. Journal of Experimental Criminology.

References

Ladenberger, N. A., & Lipsey, M. W. (2005). The Positive Effects of Cognitive-Behavioral Programs for Criminal Offenders. Journal of Experimental Criminology, 451-476.

Latessa, E. J., & Lowenkamp, C. (2006). What Works in Reducing Recidivism. University of St. Thomas Law Journal, 521-535.

Latessa, E. J., & Lowenkamp, C. (2010). The Role of Offender Risk Assessment: A Policy Maker Guide. Victims & Offenders, 203-219.

Lipsey, M. W. (2009). The Primary Factors that Characterize Effective Interventions with Juvenile Offenders: A Meta-Analytic Review. Victims & Offenders, 124-147.

Lipsey, M. W., Chapman, G. L., & Nana, A. L. (2001). Cognitive-Behavioral Programs for Offenders. The Annals of the American Academy of Political and Social Science, 144-157.

Lowenkamp, C. T., Latessa, E. J., & Holsinger, A. M. (2006). The Risk Principle in Action: What Have We Learned from 13,676 Offenders and 97 Correctional Programs? Crime & Delinquency, 77-93.

Mendel, R. A. (2011). No Place for Kids: The Case for Reducing Juvenile Incarceration. Baltimore: Annie E. Casey Foundation. Retrieved March 2018, from http://www.aecf.org/m/resourcedoc/aecf-NoPlaceForKidsFullReport-2011.pdf

Mendel, R. A. (2014). Juvenile Detention Alternatives Initiative Progress Report. Baltimore: Annie E. Casey Foundation. Retrieved March 2018, from http://www.aecf.org/m/resourcedoc/aecf-2014JDAIProgressReport-2014.pdf

References

Molm, L. D. (1988). The Structure and Use of Power: A Comparison of Reward and Punishment Power. Social Psychology Quarterly, 108-122.

Nadolny, T. L. (2017, September). How Computers are Predicting Crime - and Potentially Impacting Your Future . Retrieved from Philly Daily News: http://www.philly.com/philly/news/crime/risk-assessment-tool-criminal-justice-reform-20170921.html

Patterson, E. G. (2008). Civil Contempt and the Indigent Child Support Obligor: The Silent Return of Debtor’s Prison. Cornell Journal of Law and Public Policy, 95-141.

Pearson, J. (2004). Building Debt While Doing Time: Child Support and Incarceration. Judges’ Journal, 5-11.

Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (2006). Violent Offenders: Appraising and Managing Risk (2nd ed.). Washington: American Psychological Association.

Rice, M. E., & Harris, G. T. (2005). Comparing Effect Sizes in Follow-up Studies: ROC Area, Cohen’s d, and r. American Psychology-Law Society, 615-620.

Rice, M. E., & Harris, G. T. (2005). Comparing Effect Sizes in Follow-Up Studies: ROC Area, Cohen’s D, and r. American Psychology-Law Society, 615-620.

References

Schmidt, P., & Witte, A. D. (1988). Predicting Recidivism Using Survival Models. New York: Springer-Verlag.

Schwalbe, C. S. (2007). Risk Assessment for Juvenile Justice: A Meta-Analysis. American Psychology-Law Society, 449-462.

Schwartz, I. M., York, P., Greenwald, M., Ramos-Hernandez, A., & Feeley, L. (2016). Using Predictive Analytics and Machine Learning to Improve the Accuracy and Performance of Juvenile Justice Risk Assessment Instruments. In F. Taxman, Handbook on Risk and Need Assessment : Theory and Practice, edited by Faye S. Taxman (pp. 140-163). Routledge.

Skeem, J. L., Louden, J. E., Polaschek, D., & Camp, J. (2007). Assessing Relationship Quality in Mandated Community Treatment: Blending Care With Control. Psychological Assessment, 397-410.

Sparapani, R. A., Logan, B. R., McColluch, R. E., & Laud, P. W. (2016). Nonparametric Survival Analysis Using Bayesian Additive Regression Trees (BART). Statistics in Medicine, 2741-2753.

Stansfield, R., & Williams, K. R. (2014). Predicting Family Violence Recidivism Using DVSI-R. Criminal Justice and Criminal Behavior, 163-180.

State of Utah. (2017). Utah State Legislature. Retrieved March 2018, from H.B. 239 Juvenile Justice Amendments: https://le.utah.gov/~2017/bills/hbillenr/HB0239.pdf

References

Steinhart, D. (2006). A Practice Guide to Juvenile Detention Reform. Baltimore: Annie E. Casey Foundation. Retrieved March 2018, from http://www.aecf.org/m/resourcedoc/aecf-juveniledetentionriskassessment1-2006.pdf

Taxman, F. S., & Dezember, A. (2016). The Value and Importance of Risk and Need Assessment (RNA). In F. Taxman, Corrections & Sentencing: An Overview of the Handbook Handbook on Risk and Need Assessment : Theory and Practice. Routledge.

Ting, M. H., Chu, C. M., Zeng, G., Li, D., & Chng, G. (2017). Predicting Recidivism Amoung Youth Offenders: Augmenting Professional Judgement with Machine Learning Algorithms. Journal of Social Work, 1-19.

Ven der Put, C. E., Stams, G. J., & Dekovic, M. (2014). Assessment in the Netherlands: The Developement of a New Scoring System. Assessment, 92-107.

Waldmann, P. (2016). Genome-wide prediction using Bayesian additive regression trees. Genetics Selection Evolution.

Image Credits

Pedro Lastra

Patrick Hendry

Clem Onojeghuo

<Nelly Volkovich

End